Hierarchical Reinforcement Learning of Low-Dimensional Subgoals and High-Dimensional Trajectories
نویسندگان
چکیده
In this paper, we propose a hierarchical reinforcement learning method which enables a learner to learn tasks in a highdimensional state space. In the upper level, the learner coarsely explores the low-dimensional state space. In the lower level, the learner nely explores the high-dimensional state space. Speci cally, the learner learns to set up appropriate subgoals for the task in the upper level, and learns to achieve the subgoals in the lower level. As an example task, we choose a stand-up task involving a two-joint three-link robot. This robot has a ten-dimensional state space. The robot learns to nd subgoal postures in the upper level, and to achieve these subgoal postures in the lower level. Simulation results show that the hierarchical architecture acceralates the learning of the robot to stand up.
منابع مشابه
Hierarchical reinforcement learning with subpolicies specializing for learned subgoals
This paper describes a method for hierarchical reinforcement learning in which high-level policies automatically discover subgoals, and low-level policies learn to specialize for different subgoals. Subgoals are represented as desired abstract observations which cluster raw input data. High-level value functions cover the state space at a coarse level; low-level value functions cover only parts...
متن کاملHigh-Dimensional Unsupervised Active Learning Method
In this work, a hierarchical ensemble of projected clustering algorithm for high-dimensional data is proposed. The basic concept of the algorithm is based on the active learning method (ALM) which is a fuzzy learning scheme, inspired by some behavioral features of human brain functionality. High-dimensional unsupervised active learning method (HUALM) is a clustering algorithm which blurs the da...
متن کاملAcquisition of Stand-up Behavior by a Real Robot using Hierarchical Reinforcement Learning
In this paper, we propose a hierarchical reinforcement learning architecture for a robot with large degrees of freedom. In order to enable learning in a practical numbers of trials, we introduce a low-dimensional representation of the state of the robot for higher-level planning. The upper level learns a discrete sequence of sub-goals in a low-dimensional state space for achieving the main goal...
متن کاملSubgoal Discovery for Hierarchical Reinforcement Learning Using Learned Policies
Reinforcement learning addresses the problem of learning to select actions in order to maximize an agent’s performance in unknown environments. To scale reinforcement learning to complex real-world tasks, agent must be able to discover hierarchical structures within their learning and control systems. This paper presents a method by which a reinforcement learning agent can discover subgoals wit...
متن کاملAutonomous Subgoal Discovery and Hierarchical Abstraction for Reinforcement Learning Using Monte Carlo Method
Autonomous systems are often difficult to program. Reinforcement learning (RL) is an attractive alternative, as it allows the agent to learn behavior on the basis of sparse, delayed reward signals provided only when the agent reaches desired goals. However, standard reinforcement learning methods do not scale well for larger, more complex tasks. One promising approach to scaling up RL is hierar...
متن کامل